DIGITAL FORMATTER FOR 3 -DIMENSIONAL DISPLAY APPLICATIONS 



BACKGROUND OF THE INVENTION 

5 

1. FIELD OF THE INVENTION 

The present invention relates to 3 -dimensional (3-D) 
video displays and particularly to the digital formatters 
10 used in such displays. 

2 . DESCRIPTION OF THE RELATED ART 

3-D display applications using a sequential left- 
15 eye, right-eye technique require a high frame rate, 
usually on the order of 96-120 frames/sec, to avoid 
objectionable flicker. This rate is accomplished in some 
3-D film projection systems using two synchronized film 
projectors, one with left -eye information and the other 
20 with right -eye information, each running at a 24- 

frame/sec temporal rate, but at a 48 -frame/sec display 
(flash) rate. A correspondingly synchronized viewing 
device, typically a shutter device such as a visor, 
helmet, or goggles is worn by the viewer. The viewer- 
25 worn shutter device shutters the left-eye and right-eye 
such that a 3-D image is perceived. The viewer device 
can also be various forms of polarized elements, that 
allow light of a unique polarization to pass to each eye. 
As shown in Figure 1, for the right-eye data frame 10 and 




left-eye data frame 15, a shutter is opened twice per 
film frame with a 50% duty cycle, allowing the two 
projectors to run precisely out of phase. That is, when 
the right-eye frame Ar 11 is displayed, the corresponding . 

5 left-eye frame 16 is shuttered OFF and when the left-eye 
frame Al 17 is displayed, the corresponding right -eye 
frame 12 is shuttered OFF, The display of frames Ar and 
Al are then repeated as frames Ar 13 and Al 19, 
respectively. Again, when right -eye frame Ar 13 is 

10 displayed, the corresponding left-eye frame 18 is 
shuttered OFF, and when left -eye frame A^ 19 is 
displayed, the corresponding right-eye frame 14 is 
shuttered OFF. This effect can also be produced by 
differently polarized light used for the left and right 

15 projection combined with polarized viewing equipment. 

Using this technique., frames are processed at a temporal 
rate of 24- frames/sec and displayed twice per frame to 
provide a flash rate of 48-f rames/sec for each eye or an 
effective overall 3-D display flash rate of 96-f rames/sec 

20 for both eyes . 

Modern digital projection display systems provide 
flicker- free performance operating at 3 0 frames/sec (60- 
fields/sec, interlaced) rates, as illustrated in Figure 2. 
25 Here, the frame 20 consists of interleaved Field A 21 and 
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Field B 22, each occurring at a 60-field/sec rate. This 
means that each frame of 30 frame/sec data is flashed on 
to the screen in interleaved half fields at 60 fields/sec. 
Other displays, such as Digital Micromirror Device (DMD) 
projections displays, operate at 60 progressive (non- 
interlaced) frames/sec rates where every line is displayed 
in every frame. However, in order to avoid flicker and 
maintain a perceived fusion of motion, a 3-D version of 
such a projection display requires display rates of up to 
120-f rames/sec (twice the normal rate) due to the 
sequential left-eye, right-eye technique involved. 

Figure 3 shows the format that is typically used 
when displaying 24-frame/sec film (cinema type) media on 
a 2-D digital projection display. The 24-frame/sec film 
data 30 is converted to 60 field/sec video 31 using a 3:2 
pull -down technique as discussed in the referenced patent 
application (No. TI-26774) . Every other, frame of 24- 
frame/sec film data 30 is broken into two or three 60 
field/sec interlaced video data fields 31, respectively. 
That is, film frames A 300 and C 302 are converted to two 

video fields 3-10, a2 311 and c2 315, 316 while film 

frames B 301 and D 303 are converted to video fields B^ 

312, b2 313, b1 314, and d2 317, D^ 318, d2 319, 
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respectively. The process then repeats over and over for 
every four frames of film data 30. 

The extra field in every other frame can present 
artifacts in the projected video. This can be overcome 
by converting the 60-field/sec interlaced video 31 to 24- 
frame/sec progressive (non-interlaced) video 32. This is 
accomplished by discarding one of each extra fields of 

interlaced video, 312 or 314 and 317 or 319; 

e.g., selecting 310 (frame 1, field 1) and a2 311 

(frame 1, field 2) as progressive video frame A 320, B^ 

312 (frame 2, field 1) and 313 (frame 2, field 2), 

while discarding the data for B^ 314 (frame 3,. field 1), 

as progressive video frame B 321, 315 (frame 3, field 

2) and 316 (frame 4, field 1) as progressive video 

frame C 322, and finally 317 (frame 4, field 2) and 

318 (frame 5, field 1) , while discarding the data for 

319 (frame 5, field 2), as progressive video frame D 323. 
This data format, resulting in 24-f rames/sec of 
progressive video, can be used in each eye of a 3- 
dimensional digital projection display to provide video 
that is free of 3:2 pull-down artifacts. 

Figure 4 is a block diagram showing how video data 
is handled in a 2-D digital projection display, such as a 
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Digital Light Processor (DLP^^) projector. The system 
consists of data processing circuitry 40, which takes the 
video input signal and performs such functions as 
correction for brightness, contrast, chroma 
interpolation, and color space conversion, a digital 
formatter 41, two memory buffers 42-43, and in this case 
three Digital Micromirror Devices (DMDs) 44-46. In 
operation, while video data from one of the memory 
buffers 42 or 43 is being presented to the three DMDs for 
display, the next frame of processed video data is being 
loaded into the other memory buffer 43 or 42, 
respectively. As a result, the next frame of video data 
is always being prepared while the present frame is being 
displayed. As pointed out in the discussion of Figure 1, 
a digital 3-D display will require display rates of 96- 
120 fields/sec to avoid flicker, but because each frame 
of video is flashed twice per frame (repeated) , the 
process rate of the data need only be 48-60 fields/sec. 

The earlier invention disclosed and claimed in U.S. 
Patent Application Serial No. 09/154,461 entitled 
"Artifact Elimination for Spatial Light Modulator 
Displays of Data Converted from Film Format" is relevant 
as background information to the current invention. 
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There is a recognized need for a 3-D display, which can 
handle the bandwidth (frame rate) requirements discussed 
above at an affordable price and still provide high- 
performance video. The invention disclosed herein addresses 
this need in terms of both a method and an apparatus. 
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SUMMARY OF THE INVENTION 



This invention discloses the method and apparatus 
for a high-performance digital light processing (DLP) 3-D 
display, which has video display rates of up to 120 
fields/sec while requiring data processing rates of up to 
only 48-60 fields/sec. This approach uses quadruple 
buffering of the DLP formatter frame memory. 

In the 3-D display of this invention, two double 
buffers (quadruple memory) are used, one for the right - 
eye video data and one for the left-eye video. In a 
display formatter, a double-buffered memory is used to 
store both the frame being displayed and the frame being 
processed. This method requires essentially twice the 
memory but only half the bandwidth. As the cost of 
memory continues to decrease, this is an excellent trade- 
off to achieve high performance 3-D displays. Also, 
this technique reduces by 50% the amount of playback 
media (data) read into the system, which means that twice 
the amount of data can be put on to the playback tape or 
other storage device. 

The invention also discloses the case for playing 
back both 60 field/sec interlaced and 24 -frame/sec 
progressive video. For 60-field/sec interlaced video, a 
reverse 3:2 pull-down multiplexer is used to provide 48- 
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frame/sec, progressive data to the projector. In the case 
of the 24-f rame/sec progressive video, the 24 -frame/sec 
data from both the right-eye and left-eye are simply 
multiplexed to supply 48 -frame/sec, progressive data to 
the projector. 
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DESCRIPTION OF THE VIEWS OF THE DRAWINGS 

The included drawings are as follows: 
Figure 1 is a diagram showing how a 3 -dimensional 
5 projection display system, consisting of two 2- 

dimensional projectors synchronized to be out-of- 
phase, is read out. (prior art) 
Figure 2 is a diagram illustrating the 2:1 interleaved 
interlace technique used with 60 -field/sec video. 
10 (prior art) 

Figure 3 is a diagram illustrating the data format for 
converting 24-f rame/sec .f ilm data to 60 field/sec 
interlaced video and then to 24-frame/sec 
progressive video, (prior art) 
15 Figure 4 is a block diagram for a 2-D DLP projection 
display with three DMDs . (prior art) 
Figure 5 is a diagram showing the data format for the 3:2 
pull-down multiplexer in a 3-D DLP projection 
display. 

20 Figure 6 is a block diagram for the 3-D digital projection 
display of this invention. 
Figure 7 is a flowchart for storing the video data in the 
formatter memory of the 3-D DLP projection display 
of Figure 6 . 
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Figure 8 is a flowchart for reading out and displaying 

the video data from the formatter memory of the 3-D 
DLP projection display of Figure 6. 
Figure 9 is a timing diagram for loading and displaying 
video data from the formatter memory of the 3-D DLP 
projection display of Figure 6. 
Figure 10 is a diagram showing the data format for 
playing back (displaying) data, which has been 
captured at 24 -frames/sec and converted to 60 
field/sec interlaced video. 
Figure 11 is a block diagram for the reverse 3:2 pull- 
down multiplexer used in the data conversion 
technique shown in Figure 10 . 
Figure 12 is a flowchart showing how video data is loaded 
into the FIFO field buffers of the reverse 3:2 pull- 
down multiplexer of Figure 10 . 
Figure 13 is a flowchart showing how data is read out of 
the FIFO field buffers, of the reverse 3:2 pull-down 
multiplexer of Figure 10. 
Figure 14 is a timing diagram for loading and reading out 
the FIFO field buffers of the reverse 3:2 pull-down 
multiplexer of Figure 10. 
Figure 15 is a block diagram for the 3-D projection 

display of this invention with direct playback of 
video, which has been captured at 24-f rames/sec . 
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DETAILED DESCRIPTION 

This invention discloses a method and apparatus for 
a 3-D digital projection display. The projector uses a 

5 quadruple memory buffer to store and read processed video 
data for both the right -eye and left -eye display. Video 
data can be processed at a rate of 30-60 frames/sec, 
which is within the bandwidth requirements of typical 2-D 
displays of this type, and displayed with a flash rate of 

10 60-120 frames/sec. 

The diagram of Figure 5 shows the video data format 
for the 3-D digital projection display of this invention. 
Right-eye video 50 and left-eye video 52, at 60 interlaced 
15 fields/sec, is converted to progressive 24-frame/sec video 
for both right-eye 51 and left-eye" 53, respectively. The 



^'i data is then combined to provide 96-f rames/sec video 54 by 

•i 

l^i sequentially reading and then repeating again each frame 

h 

■ of right-eye 51 and left-eye 53 data. The method used for 
20 converting the data for each eye is that described earlier 
in Figure 3. The method of this invention takes the 24- 
frame/sec data from each eye and sequentially combines it 
at 48-f rames/sec and then repeats it again to provide 96- 
f rames/sec' video. The process uses a reverse 3:2 pull- 
25 down technique to convert the right-eye interlaced 60 
field/sec frame A field 1 A^^ 500 and field 2 A^^ 501, 
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frame B field 1 B^r 502 and field 2 B^j^ 503, frame C field 
2 C2j^ 505 and field 1 C^r 50S, and frame D field 2 D^j^ 507 
and field 1 D^^ 508 into right-eye, 24-f rames/sec 
progressive video Ar 510, Br 511, Cr 512, and Dr 513. 
5 Right-eye fields B^r 504 and D^r 509 are referred to as 

redundant fields and the data from these are discarded and 
not used in this process. Alternately, frame B field 1 
B^R 504 and frame D field 2 D^r 509 could be selected, 
thereby making frame B field 1 B^r 502 and frame D field 2 
10 D^R 507, respectively, the redundant fields. The same 
process is used to convert the left-eye interlaced 60 
field/sec frame A field 1 A^l 520 and field 2 A^^ 521, 
frame B field 1 B^^ 522 and field 2 B^^ 523, frame C field 
2 525 and field 1 C^^ 526, and frame D field 2 D^^ 527 

15 and field 1 D^l 528 into left-eye, 24-f rames/sec 

progressive video A^ 530, B^ 531, Cl 532, and 533. 
Left-eye fields B^l 524 and D^^ 529 are also redundant 
fields and the data from these are discarded and not used 
in this process. As before, frame B field 1 B^l 524 and 
20 frame D field 2 D^^ 529 could be selected, thereby making 
frame B field 1 B^l 522 and frame D field 2 D^^ 527, 
respectively, the redundant fields. Finally, the 24- 
frame/sec right-eye 51 and left-eye 53 data is 
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sequentially combined and repeated to provide 96 -frame/sec 
progressive video 54 . This whole sequence is then 
repeated over and over to provide the 3-D video data 
stream . 

5 

Figure 6 is a block diagram 60 for the 3-D projection 
display of this invention. The system consists of data 
processing circuitry 61, which takes the video input 
signal and performs such functions as correction for 
10 brightness, contrast, chroma interpolation, and color 
space conversion, a digital formatter 62, four memory 
buffers 63-66, and three digital micromirror devices 
(DMDs) 67-69. In this case, the memory buffers have been 
expanded from the two buffers typically used in a 2-D 
iq 15 projection display to four buffers 63-66 for the 3-D 
Q display. There are two double-buffers 63,65 and 64,66, 

|i| which are used to alternately store and display processed 

Li: 

O data; e.g., each double-buffer 63,65 or 64,66 contains the 

right-eye and left-eye data, respectively. Progressive 
20 input data is supplied to be processed by the data path- 
processing block 61. In operation, while right-eye and 
left-eye data for frame A is being displayed from buffer 
64 and buffer 66, respectively, processed right-eye and 
left-eye data for field B are stored in buffer 63 and 
25 buffer 65, respectively. By doubling the memory buffer in 
this method, video can be displayed at flash rates of up 
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to 120 -frames/sec, which is twice the normal rate, to 
present a flicker-free 3-D picture. 

Figure 7 is a flowchart showing how data is loaded 
5 into the four formatter memory buffers 63-66 of Figure 6. 
The even-odd frame decision block 70 determines if the 
processed data is from an even numbered frame of the 
progressive 48-frame/sec video. If YES (it is from an 
even frame) , then the buffer 1 right-eye decision block 
10 71 determines if it is right-eye video. If YES (it is 
right-eye video) , then the data is stored in the right- 
eye buffer 1 (RE-BUF 1) 72 and if NO (it is not right-eye 
video) , the data is stored in the left-eye buffer 1 (LE- 
BUF 1) 73 and then in either case the buffer loading 
15 cycle repeats. On the other hand, if the even frame 
f-% decision block 70 decision is NO (it is not from an even 

y frame) , then the buffer 2 right-eye decision block 74 

Q determines if it is right-eye video. If YES (it is 

* right-eye video) , then the data is stored in the right- 
20 eye buffer 2 (RE-BUF 2) 75 and if NO (it is not right-eye 
video) , the data is stored in left-eye buffer 2 (LE-BUF 
2) 76 and then in either case the buffer loading cycle 
repeats. 

25 Figure 8 is a flowchart showing how data is read out 

and displayed from the four formatter memory buffers 63- 
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66 of Figure 6. Data from each buffer is displayed on 
the screen twice for each eye with the right-eye and 
left-eye data being 180*^ out of phase, so that while data 
is displayed to one eye, the other eye is blacked out by 

5 a visor or similar device. As discussed earlier, while 
data from one buffer are being displayed, processed data 
is being stored in the other buffer. Frames of data are 
sequentially read from buffers RE-BUFl 80 and LE-BUFl 81, 
repeated from these buffers RE-BUFl 82 and LE-BUFl 83, 

10 and then read from buffers RE-BUF2 84 and LE-BUF2 85, and 
repeated from these buffers RE-BUF2 86 and LE-BUF2 87. 
This readout sequence is then repeated over and over. 
Although the data is processed at 24-f rames/sec for each 
eye and each buffer is displayed on the screen at 48- 

15 frames/sec, the combining of the right-eye/left-eye video 
increases the effective display rate to 96-f rames/sec . 

Figure 9 is a timing diagram for loading and 
displaying data from the formatter memory buffers 63-66 of 

20 Figure 6. This illustrates how one of the right-eye or 
left-eye frame buffers gets loaded in 1/48 sec (for 
example RE-BUFl 6-4 with right-eye frame A data) so that a 
frame of right-eye/left-eye data is loaded into the 
buffers in 1/24 sec (for example RE-BUFl and LE-BUFl) . 

25 Data is then readout from the buffers and displayed twice 
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for each frame in 1/24 sec (equivalent of displaying data 
from each eye in 1/96 sec) . This diagram also illustrates 
that while data is being processed and stored in one 
buffer (example Cr/Cl) it is being displayed twice from 
the other buffer (example E^/Bj_^) in the same period of 
time. 

Playback source data is often captured at 24- 
frames/sec and converted to 60 fields/sec by applying a 
3:2 pull-down technique. Figure 10 is a block diagram 
illustrating the sequence for the playback of data, such 
as film 100 (media) , which has been captured off-line at 
24-f rames/sec and converted to 60 interlaced fields/sec by 
applying a 3:2 pull-down technique. The original data is 
recorded from two sources, such as cameras, located in 
space such as to represent the right and left eyes, 
respectively. The captured data 100 is fed into right-eye 
101 and left-eye 102 playback devices. The sequence of 
the right-eye playback device 101, still at 60 fields/sec, 
is 

aIr a2r bIr b2r bIr c2r C^r d2r D^r d2r and 
of the left-eye playback device 102 

A^L a2l B^l b2l bIl C^j, C^l 02^ D^l ^^L- 
These two outputs signals are fed into the field buffers 
of the reverse 3:2 pull -down multiplexer 103 where field 
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1 and field 2 are interleaved, the right-eye and left-eye 
data is multiplexed, and the redundant fields are removed 
to provide 48-f rames/sec progressive video with the 
following data sequence: 

Ar Al Br Bl Cr Cl Dr Dl . 
The data is then provided to the projection display 60 
(discussed earlier in Figure 6) for display. 

Figure 11 is a block diagram for the internal 
workings of the reverse 3:2 pull-down multiplexer 103 
shown in Figure 10. This circuitry is comprised of right 
field and left field buffers 110,111 and a multiplexer 
(MUX) 112, and is used to convert the 50- field/sec 
interlaced video from the playback devices 101,102 into 
48-frame/sec progressive data. The two field buffers 
110, 111 consist of first-in/first-out (FIFO) field 1 
buffers RFBl 1101, LFBl 1104 and field 2 buffers RFB2 
1102, LFB2 1105 and multiplexers 1103,1106 to handle the 
right and left field data, respectively. Although the 
output data rate of each 3:2 pull-down frame buffer 
110,111 is 24 progressive frames/sec, each frame buffer 
output is displayed in 1/48 th sec so as to interleave 
with the other frame buffer output. The output of the 
right field buffer 110 is the 24-frame/sec progressive 
signal 
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Ar Br Cr Dr, and 
the output of the left field buffer 111 is the 24- 
frame/sec progressive signal 

Al Bl Cl Dl 

These two 24-f rame/sec progressive signals, originating 
from the right-eye and left-eye playback devices 101,102, 
respectively, are then combined in a ping-pong fashion by 
multiplexer 112 to provide the 48-frame/sec progressive 
signal 

Ar Al Br Bl Cr Cl Dr Dl 
to the projection display. ' 

Figure 12 is a flowchart showing how right -eye and 
left-eye data is stored in the field buffers 110,111 of the 
i^everse 3:2 pull-down multiplexer 103. Both the right and 
left field buffers 110 and 111 process data in parallel so 
that this flowchart applies to both. First, decision block 
120 determines if the data is from a redundant field. If 
YES (it is from a redundant field) , the data is discarded 
and the cycle repeats. If NO (it is not from a redundant 
field) , then the decision block 121 determines if this is 
field 1 data. If YES (it is field 1 data) , then the data 
is stored in the field 1 FIFO buffer 122 (example, RFBl) 
and the cycle repeats. If NO (it is not field 1 data), the 
data is stored in the field 2 FIFO buffer 123 (example, 
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RFB2) and the cycle repeats. This process places the video 
field data in buffers RFBl 1101 and RFB2 1102 for right-eye 
data and in buffers LFBl 1104 and LFB2 1105 for left-eye 
data. Notice that the sequence of fields for the data at 
the input of the field buffers 110,111 changes in mid- 
stream; i.e., for example, in the case of the right-eye it 
is field 1, field 2, field 1, field 2, and then field 2, 
field 1, field 2, field 1, as follows 
Alp a2r bIr b2r I c2p Clp D2p 



'R ^^R ^^R ^^R 



10 This is caused by the removal of the redundant data, 
p However, the data gets put into the field memory FIFO 

\[| buffers 1101 and 1102 in correct order, 

-* 

4^ Figure 13 is a flowchart for reading data out of the 

W 15 right and left field buffers 110 and 111 into the 3:2 

pull-down multiplexer 112. The 24-frame/sec progressive 
data is read from the right field 1 and right field 2 
FIFO buffers 1101, 1102 through multiplexer 1103. 
Similarly, the 24-frame/sec progressive data is read from 
20 the left field 1 and right field 2 FIFO buffers 1104, 

1105 through multiplexer 1106. The flowchart applies to 
both the right and left field buffers 110 and 111. 
In the flowchart, lines are read in sequence from field 
buffer 1 block 130 (example: from RFBl 1101) and field 
25 buffer 2 block 131 (example: from RFB2 1102) . Then 
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decision block 132 determines if this was the last line 
of video in the frame. If NO (not last video line), the 
next pair of lines from the two field buffers are read, 
and so on until decision of block 132 is YES (is last 
line) at which point the next frame of video readout is 
started. Relating back to Figure 11, this relates to 
alternately reading the RFBl 1101 (example A^r) and RFB2 
1102 (example A^j^) to give a 24 -frame/sec progressive 
frame of video, followed by alternately reading lines 
from the LFBl 1104 (example A^^) and LFB2 1105 (example 
A^l) give a second 24 -frame/sec frame of progressive 
video and so on. 

Figure 14 is a timing diagram illustrating how data 
is handled in the playback system of Figure 9. As shown, 
data is fed simultaneously into the right and left field 
buffers 110,111 in phase at 1/60 sec per field and then 
read out of the f ield * buf f ers into multiplexer 112 in a 
ping-pong fashion at 1/48*^^ sec per frame to provide the 
48-frame/sec progressive video format of 



discussed earlier in Figure 11. 

There is an ever increasing desire to capture and 
directly playback 24 -frames/sec progressive video through 



Ar Al Br Bl Cr Cl Dr Dl , 
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a projection display. Figure 15 is a block diagram 
showing the- case where 24 -frames/sec progressive video is 
fed directly into the display system. The system 
consists of the 24-frame/sec input source 150 (media) , a 
5 right-eye playback device 151, a left-eye playback device 
152, a right field buffer 153, a left field buffer 154, a 
multiplexer 155, and a projection display 60 (from Figure 
6) . In comparing Figure 15 with Figure 10, it is seen 
that the reverse 3:2 pull-down multiplexer is no longer 
10 required in this case since there is no 60 field/sec 

interlaced video and as a result the circuitry is simpler 
and more straightforward. However, the right and left 
frame buffers 153, 154 and MUX 155 are still required to 
alternate the 24-f rames/sec data and provide the 48- 
15 frame/sec progressive video to the projection display 
system 60 . 

The descriptions of various embodiments above 
include assorted means of converting frame rates from a 
transmitted frame rate to a native frame rate at which 

20 the video content was first captured. It should be 

recognized that coupling the core techniques described 
herein with some video source eliminates the need for 
these various video frame rate conversions. For example, 
one embodiment of the present invention receives data as 

25 a dual 24 frame/second video steam (24 first eye and 24 
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second eye frames per second) . The data is stored in the 
double-buffered memory and read out twice (first-eye, 
second-eye, first-eye, second-eye) to achieve a 96 
frame/second display rate. Likewise, data received at a 

5 dual 30 frame/second frame rate can be stored and 

displayed twice to achieve a 120 frame/second display 
rate. These various embodiments provide the advantage of 
producing a high display frame rate without requiring a 
high bandwidth processor. Data is processed at half the 

10 rate at which it is displayed, and merely displayed twice 
each frame. 

While this invention has been described in the 
context of a preferred embodiment, it will be apparent to 

15 those skilled in the art that the present invention may be 
modified in numerous ways and may assume embodiments other 
than that specifically set out and described above. 
Accordingly, it is intended by the appended claims to 
cover all modifications of the invention that fall within 

20 the true spirit and scope of the invention. 
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